fix: keyword list parsing crash #56
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mhanberg
reviewed
Feb 7, 2025
lib/spitfire.ex
Outdated
Comment on lines
557
to
559
| parser = parser |> next_token() |> next_token() | ||
| {item, parser} = parse_expression(parser, @kw_identifier, true, false, false) | ||
| {item, parser} |
Contributor
There was a problem hiding this comment.
i believe this case actually should result in an error,
iex(1)> Code.string_to_quoted("[foo: \"bar\", :baz]")
{:error,
{[line: 1, column: 12],
"unexpected expression after keyword list. Keyword lists must always come last in lists and maps. Therefore, this is not allowed:\n\n [some: :value, :another]\n %{some: :value, another => value}\n\nInstead, reorder it to be the last entry:\n\n [:another, some: :value]\n %{another => value, some: :value}\n\nSyntax error after: ",
"','"}}
mhanberg
reviewed
Feb 7, 2025
Comment on lines
+538
to
+546
| code = ~S''' | ||
| defmodule MyModule do | ||
| IO.inspect( | ||
| :stderr, | ||
| label: "label", | ||
| (__cursor__()) | ||
| ) | ||
| end | ||
| ''' |
Contributor
There was a problem hiding this comment.
this is an interesting result from the builtin parser
iex(5)> Code.string_to_quoted(~S'''
...(5)> defmodule MyModule do
...(5)> IO.inspect(
...(5)> :stderr,
...(5)> label: "label",
...(5)> (__cursor__())
...(5)> )
...(5)> end
...(5)> ''')
{:ok,
{:defmodule, [line: 1],
[
{:__aliases__, [line: 1], [:MyModule]},
[
do: {{:., [line: 2], [{:__aliases__, [line: 2], [:IO]}, :inspect]},
[line: 2], [:stderr, [{:label, "label"}, {:__cursor__, [line: 5], []}]]}
]
]}}
iex(6)> Code.string_to_quoted(~S'''
...(6)> defmodule MyModule do
...(6)> IO.inspect(
...(6)> :stderr,
...(6)> label: "label",
...(6)> :foo
...(6)> )
...(6)> end
...(6)> '''
...(6)> )
{:error,
{[line: 4, column: 19],
"unexpected expression after keyword list. Keyword lists must always come as the last argument. Therefore, this is not allowed:\n\n function_call(1, some: :option, 2)\n\nInstead, wrap the keyword in brackets:\n\n function_call(1, [some: :option], 2)\n\nSyntax error after: ",
"','"}}
iex(7)>
Contributor
Author
There was a problem hiding this comment.
Either the builtin parser supports a macro call returning tuple AST node or this is an error
Contributor
Author
There was a problem hiding this comment.
Another possibility it is more forgiving for nodes with __cursor__() special form
Contributor
Author
There was a problem hiding this comment.
@mhanberg We have two options here.
- Have a special handling for
__cursor__()nodes in keyword lists - Make the parser error tolerant and either silently drop invalid list entries or produce invalid AST
lukaszsamson
force-pushed
the
ls-fix-unfinished-keyword
branch
from
b169652 to
41c96eb
Compare
|
I was trying to integrate Spitfire into Expert and run into the same issue this PR is trying to fix, triggered by code like |
Add test cases from elixir-tools#32
mhanberg
force-pushed
the
ls-fix-unfinished-keyword
branch
from
41c96eb to
a0842be
Compare
mhanberg
reviewed
Jan 6, 2026
Comment on lines
+2941
to
+2952
| @tag foo: bar, | ||
| foo | ||
| """ | ||
|
|
||
| assert Spitfire.parse(code) == | ||
| { | ||
| :ok, | ||
| {:@, [end_of_expression: [newlines: 1, line: 2, column: 6], line: 1, column: 1], | ||
| [ | ||
| {:tag, [line: 1, column: 2], | ||
| [[{:foo, {:bar, [line: 1, column: 11], nil}}, {:foo, [line: 2, column: 3], nil}]]} | ||
| ]} |
Contributor
There was a problem hiding this comment.
Sorry for taking a year to dive into this more.
Shouldn't this be an error in this case?
mhanberg
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes crash 3 from #48
Fixes #26
Supersedes #32